================================================================================
ANNOTATION GUIDELINES
Stage 2: Requirement Derivation and Explanation Generation
Reg2Req Project | Version 1.0
================================================================================


1. PURPOSE AND SCOPE
--------------------------------------------------------------------------------
These guidelines describe how to evaluate derived software requirements and their 
accompanying plain-language explanations.

For each evaluation item you will receive:
  - A source regulatory clause 
  - One or more software requirements derived from that clause by the pipeline
  - A plain-language explanation for each derived requirement

You will rate each requirement and its explanation on three dimensions each,
using a 1-5 scale with rubric-anchored criteria. You will also provide a
binary plausibility rating for each explanation.

  [NO LEGAL EXPERTISE REQUIRED]
  You do not need legal expertise to complete this evaluation. The metrics are
  designed to be assessed from the perspective of a software engineer or
  requirements engineer who will need to act on the derived requirements. When
  in doubt, ask: would a competent software engineer understand and be able to
  implement this requirement correctly?

2. REQUIREMENT EVALUATION METRICS
--------------------------------------------------------------------------------
Rate each derived requirement individually on Correctness and Clarity. Rate
the full set of requirements derived from a single clause collectively on
Completeness.

  CORRECTNESS (per-requirement, scale 1-5)
  What to assess: Does the requirement accurately reflect the clause content
  without introducing unsupported statements or over-generalizing? Penalize
  fabricated constraints or misrepresented scope.

    1 = Substantially incorrect; contains fabricated or contradictory content
    2 = Partially correct; contains notable inaccuracies
    3 = Mostly correct; minor inaccuracies
    4 = Correct with negligible issues
    5 = Fully correct; accurately reflects the clause

  COMPLETENESS (per-clause, scale 1-5)
  What to assess: Considering all requirements derived from this clause
  collectively -- have all distinct behaviors, interactions, and properties
  implied by the clause been captured? Score the full set, not individual
  requirements.

    1 = Major behaviors missing; clause substantially under-covered
    2 = Some important behaviors missing
    3 = Most behaviors captured; minor omissions
    4 = Nearly complete; trivial omissions only
    5 = All implied behaviors captured

  CLARITY (per-requirement, scale 1-5)
  What to assess: Is the requirement unambiguous and comprehensible to a
  software engineer without legal training? Penalize requirements that copy
  legal phrasing verbatim without translation.

    1 = Incomprehensible or highly ambiguous
    2 = Difficult to understand; significant ambiguity
    3 = Understandable with effort; some ambiguity
    4 = Clear; minor wording issues
    5 = Fully clear and unambiguous

  [COMPLETENESS RATING NOTE]
  Completeness is always a per-clause rating. When a clause has multiple
  derived requirements, read all of them together and then assign a single
  completeness score for the clause. Do not assign separate completeness
  scores to individual requirements.


3. EXPLANATION EVALUATION METRICS
--------------------------------------------------------------------------------
Rate the explanation for each derived requirement on the following four
dimensions:

  CORRECTNESS (per-explanation, scale 1-5)
  What to assess: Does the explanation accurately reflect the link between the
  source clause and the derived requirement? Penalize explanations that
  misattribute the regulatory intent or link the wrong clause content to the
  requirement.

    1 = Substantially incorrect; explanation contradicts or misrepresents
        the clause
    2 = Partially correct; notable inaccuracies in linking clause to
        requirement
    3 = Mostly correct; minor inaccuracies in linking
    4 = Correct with negligible issues
    5 = Fully correct; accurate and precise connection

  COMPLETENESS (per-explanation, scale 1-5)
  What to assess: Does the explanation cover all reasoning linking the source
  clause to the derived requirement? An explanation that omits material
  conditions, exceptions, or justifications should be penalized.

    1 = Missing major reasoning steps
    2 = Some important reasoning present but significant gaps remain
    3 = Most reasoning present; minor gaps
    4 = Nearly complete; trivial omissions only
    5 = Full reasoning chain present

  CLARITY (per-explanation, scale 1-5)
  What to assess: Is the explanation comprehensible to a requirements engineer,
  developer, tester, or auditor without legal training? The explanation should
  translate legal language into engineering terms.

    1 = Incomprehensible to a non-lawyer
    2 = Difficult to understand; significant legal terminology unexplained
    3 = Understandable with effort; some legal terms not translated
    4 = Clear; minor wording issues or isolated unexplained terms
    5 = Fully accessible; no legal knowledge required

  PLAUSIBILITY (per-explanation, binary 0/1)
  What to assess: Is the explanation reasonable and supported by the
  requirement? This is a binary assessment: the explanation should not be
  implausible or internally inconsistent, even if incomplete.

    0 = Implausible or internally inconsistent
    1 = Plausible and internally consistent

  [PLAUSIBILITY VS. QUALITY]
  Plausibility is a binary check, not a quality rating. It simply asks: is
  the explanation internally consistent and not contradictory? Even a
  low-quality explanation can be plausible. An implausible explanation would
  be one that states something logically contradictory, e.g. claiming the
  clause requires data deletion and then explaining that the system should
  retain the data.


4. ANNOTATION PROCEDURE
--------------------------------------------------------------------------------
  1. Read the source clause carefully.
  2. Read all derived requirements for that clause.
  3. Rate each requirement individually on Correctness (1-5) and
     Clarity (1-5).
  4. Rate the full set of requirements for the clause collectively on
     Completeness (1-5).
  5. Read the explanation for each requirement.
  6. Rate each explanation on Correctness (1-5), Completeness (1-5),
     Clarity (1-5), and Plausibility (binary 0/1).
  7. Record your ratings in the provided spreadsheet. Do not leave any
     cells blank.
  8. If you are uncertain about any rating, flag it and note your uncertainty
     in the comments column.
